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This paper is one of four individual case study reports presenting the qualitative 
findings from a five-year investigation of the impact of accountability reform on students 
with disabilities in four states, eight districts, and twenty schools. During the late 1980s 
and 1990s, many states passed legislation requiring school systems to be more 
accountable for students’ learning. However, the impetus for the most recent changes in 
state accountability systems originated in state responses to federal concerns about the 
quality and equity of public education. These concerns can be considered as the next 
phase of the standards-based reform movement. 

The standards-based reform movement, which began during the 1970s, has been 
integrated into federal education policy, most notably the last two reauthorizations of the 
Elementary and Secondary Education Act (ESEA), and has become increasingly focused 
on holding schools accountable for the academic performance of all their students. The 
drive for greater accountability and educational equity is embedded in the 2001 
reauthorization of ESEA, known as the No Child Left Behind Act (NCLB). NCLB aims 
to increase student achievement, improve schools, provide parents and the community 
with better information, and close some long-lasting and troubling achievement gaps 
between disadvantaged students and their peers (Cohen, 2002, p. 61). 

The new requirements for standards and assessments are rigorous and largely 
build on the existing Title I requirements promulgated under the Improving America’s 
Schools Act of 1994. For most states however, performance-based accountability 
requires them to chart a course into new and unfamiliar territory. Particularly 
controversial is the requirement that the same annual academic achievement objectives be 
established, met, and reported for specific subgroups, including most students with 
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disabilities, racial/ethnic minority groups, and students with limited English proficiency 
(LEP). 

This case study report presents qualitative data collected over a four-year 
period — 2001-2004 — from three levels of the public education systems, and provides an 
analysis of the impact of accountability reform on students with disabilities in one of our 
participating four states. The following three research questions guided this component of 
the project: 

1 . How do broad education policies that incorporate high-stakes accountability 
include consideration of students with disabilities? 

2. How are students with disabilities affected by education accountability 
reforms? 

3. What impact have educational accountability mechanisms had on students 
with disabilities at the system and individual student level? 

The remainder of this report is divided into five sections. Section I contains 
information regarding the methodology used to collect and analyze the state and local 
data presented in this report. Section II is an overview of the assessment and 
accountability systems in Maryland, including changes resulting from NCLB. In Section 
III we provide demographic information on each of the study sites. In Section IV we 
present our research findings at the state, district, and school level, and in Section V we 
present a discussion of the major findings for each embedded case study and address 
cross-site themes. 



3 




I. METHODOLOGY 



Identification of Study Sites and Data Sources 

The Education Policy Refonn Research Institute (EPRRI), funded by the United 
States Department of Education’s Office of Special Education Programs (OSEP), was 
established in 2000 to investigate and describe the impact of including students with 
disabilities in new state educational accountability systems as required by the 1997 
reauthorization of the Individuals with Disabilities Education Act (IDEA 1997). EPRRI 
researchers adopted an empirical approach to answering the research problem — the case 
study method — and specified a priori the levels of analysis, participants, and questions to 
be pursued. One prerequisite for selection was the requirement that study states have high 
quality data collection systems that could provide disaggregated data for students with 
disabilities. 

Four core study states agreed to participate in the five year study: California, 
Maryland, New York, and Texas. Within each state, EPRRI staff and core state 
representatives identified and secured the participation of two school districts in the 
study. The four selected states varied across several key accountability features, including 
high-stakes versus low-stakes accountability consequences, recentness of reforms, 
stability versus instability of reform efforts, participation of students with disabilities in 
all accountability reports, and use of alternate assessments. In addition, the sites also 
reflected geographic and demographic diversity. It is important to recognize that in 
December 2001 the policy context surrounding this study changed substantially with the 
passage of NCLB and as a result the study sites became more homogeneous in their key 
accountability features. Unfortunately, due to our initial study design we had already 
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selected sites and participants on an a priori basis (versus selecting them purposively and 
serially over time) and it is possible that we limited our opportunities to study relevant 
issues that only became apparent after NCLB and over the course of the study. Despite 
these reservations, we are confident that our analysis of data obtained from selected 
states, districts, and schools reveals important issues and key challenges as states, 
districts, and schools nationwide work to meet the ongoing requirements of NCLB. 

EPRRI researchers used two complementary strategies to collect qualitative data: 
analysis of documents and in-depth interviewing (Marshall & Rossman, 1999). The use 
of two methods of data collection and multiple data sources allowed for triangulation by 
data source. EPRRI staff developed a sound understanding of the history and context of 
educational reform in each of the study sites by ongoing reviews of state and district 
policies available online or obtained directly from the participants. Examples of the types 
of state documents reviewed include Board of Education policies and minutes, strategic 
plans, reports from Superintendents and Commissioners, and state Education Department 
letters to districts. We also reviewed state-consolidated applications for federal funding 
under NCLB and state accountability plans submitted to the U.S. Department of 
Education to comply with NCLB. In addition we collected press releases and letters 
issued by the Secretary of Education, and speeches by the President and Secretary that 
were obtained from the U.S. Department of Education web site. We also reviewed reports 
issued by the U.S. General Accounting Office on NCLB, journal articles and local 
newspaper articles from across the four sites. Finally, we monitored state Web sites for 
information on evolving state efforts to implement NCLB and for state policy documents 
developed to meet the NCLB requirements. 
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The guiding questions for the research came directly from OSEP’S grant priority, 
but were further broken into subheadings and sample issues/indicators. In February 2001, 
the state-level participants and EPRRI staff together reviewed and revised a draft 
template of research questions. This template is available on the EPRRI website, 
www.eprri.org . This process led to the development of 10 individual interview protocols 
focused on the areas of accountability, assessment, monitoring, curriculum and 
instruction, special education, teacher certification, professional development, Title I, 
transition, and parent viewpoints. In October 2001, the same process was used to develop 
interview protocols at the district level. Individual school level protocols were developed 
for the school principal, regular education teachers and special education teachers in late 
2002 . 

Selection of participants at the state and district level was done purposively with 
the support and cooperation of the state and district directors of special education, who 
acted as gatekeepers. At the state and district levels, the special education director 
participated in the identification, initial contact, and interview arrangements with key 
personnel. Knowledgeable personnel from the following departments in each SEA and 
LEA were interviewed: special education, accountability, testing, special education 
monitoring, Title I monitoring, curriculum, teacher certification, and professional 
development. 

EPRRI was specifically concerned with identifying schools within its study sites 
that were high perfonning for students with disabilities compared to other schools in the 
same district. However, the tenn “high performing” is used relatively, as in general the 
performance of students with disabilities lagged behind that of students without 
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disabilities by an average of 30 percentage points. We used one of two ways to identify 
each high-performing school: (a) its school perfonnance index (SPI) or (b) district 
recommendation. 

School Performance Index. The SPI allowed a closer examination of the 
elementary schools in our participating districts and identified schools that were 
relatively high performing for this population. Several factors were considered important 
in the creation of this index. First, we wanted to capture performance across multiple 
years, to minimize the chance of misinterpreting anomalies in perfonnance data. 
Fluctuation in data is especially an issue when examining performance trends in schools 
with low numbers of special education students. In addition to considering performance 
over more than one year, we wanted to include performance across more than one grade 
level and across both reading/language arts and math. By including multiple indicators of 
success, we created an index reflective of “high performance” with respect to multiple 
domains. Most importantly, the index allowed us to identify schools that demonstrated 
some degree of success with all students over a minimum of two successive school years. 

The first step in creating the school index was to sum the performance data for the 
various performance variables described above. In this manner, performance data for 
students with disabilities were weighted equally to performance scores for students in 
general education. We viewed this as appropriate give our specific interest in pinpointing 
and examining schools that demonstrate some level of relative success for students with 
disabilities in addition to some level of overall success. Summed performance scores 
were then standardized to allow for comparison across states. 
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Recognizing the strong correlation between socioeconomic status and 
performance on standardized assessments, we wanted to take the overall socioeconomic 
status of each school’s population of students into consideration when we examined 
index scores. Therefore, a variable reflecting each school’s socioeconomic status 
(School-SES) over the data collection years included in each state’s school perfonnance 
index was also created. The “School-SES” variable was operationalized as the mean 
percent of students receiving free or reduced price meals at each school over the two data 
collection years included in the school performance index, which was then standardized 
and reported as a z-score. Schools with performance index scores above the mean (i.e., z- 
scores at or above zero) that were also above the trend line reflecting the overall 
correlation of school SES to performance were selected for interviews. Schools in 
Maryland, New York, and Texas were selected in this manner. 

Overview of Data Collection Procedures 

During this five-year multi-state study we conducted 35 interviews at the state 
level and 44 at the district level. State level interviews occurred between October 2001 
and January 2002 and district level interviews took place between April and June 2002. 
Once we had identified the schools, the special education director from each LEA 
facilitated the initial contact and we worked closely with the school principal to 
determine when it would be convenient to visit. School visits occurred between 
December 2003 and June 2004. We conducted in-depth interviews with special and 
general educators and the school principals from 20 elementary schools. We requested 
that the individual school principals select teachers for interviews based on the following 
criteria: (a) he or she had worked in the school for two or more years, (b) he or she 
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worked with students in the grades assessed by their state for the purpose of 
accountability, and (c) the general education teacher had students with disabilities in his 
or her classroom during the current academic year. 

The interviews at all levels of the education system were between one and two 
hours in length and adopted a semi-structured approach with the interview protocols 
acting as rough guides. Interviews were also audiotaped with the participants’ consent so 
that they could take the form of a conversation. Finally, the interviews were conducted by 
multiple researchers (Eisenhardt & Bourgeois, 1988), allowing the researchers to overlap 
data analysis and data collection, as recommended by Glaser and Strauss (1967) as well 
as Van Maanen (1988). The use of multiple investigators provided complementary 
insights, added richness to the data analysis, and enhanced confidence in the findings 
(Eisenhardt, 2002). 

Data Analysis 

EPRRI researchers followed the qualitative data analysis procedures 
recommended by Miles and Huberman (1994). After each interview field notes were 
written up by individual researchers, while the taped interviews were transcribed by 
graduate students. At each level of the system individual case study data analysis began 
with the creation of a contact summary sheet to develop an overall picture of the main 
points of each interview. The graduate assistant who first transcribed the interview then 
read each contact summary sheet to identify bias and selectivity. We developed detailed 
descriptive write-ups for each site, based on the field notes, the contact summary sheet, 
and the document reviews. This step in the data analysis is central to the generation of 
insight and helps researchers to cope with the enormous volume of data (Miles & 
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Huberman, 1994; Pettigrew, 1988). 

We then entered the transcripts into a qualitative software program, Ethnograph, 
which allows for the analysis of text-based data into codes and categories of meaning. 
EPRRI researchers adopted a coding approach partway between the a priori and inductive 
approaches discussed by Miles and Huberman (1994). A general coding scheme, based 
on that developed by Bogdon and Biklen (1992), was created to provide a structural, 
conceptual, and coherent order to the emerging codes. This particular coding scheme was 
based on the general domains covered by the research matrices and the interview 
protocols and allowed codes to develop inductively, while at the same time enabling the 
researcher to “think about categories in which codes will have to be developed” (Miles & 
Huberman, 1994, p. 61). 

We created clear operational definitions for each code so that the codes could be 
applied consistently. Code names closest to the concepts being described were applied to 
the chu nk s of data. Initial coding of the data corpus was performed by the first author and 
a team of graduate research assistants, who read and reread each interview line by line 
and coded the sentences or phrases relating to the participants’ perceptions of the effects 
of accountability on students with disabilities and the systems that serve them. EPRRI 
researchers followed the recommendations of Miles and Huberman (1994) for check- 
coding. Two researchers separately coded the first eight pages of the opening interviews 
from each level of the system and reviewed the coded sections together. Intercoder 
reliability was determined using the following fonnula: 

Reliability = number of agreements / total number of agreements + disagreements. 
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Initial intercoder reliability at each level ranged from 79 to 83 percent and rose to 
between 9 1 and 93 percent after the differences were clarified. A further check was 
performed two-thirds of the way through the data analysis. A conceptually clustered 
matrix was developed during the early analysis based on the interview protocols. The 
following decision rules were applied as themes were identified: (a) a theme was coded 
as present for a participant if it was mentioned repeatedly or with strong emphasis during 
the interview; and (b) a theme was coded as present for a study site if it was mentioned 
by two or more participants. 

The second step in the analysis was to put the data back together again in a new 
way to reveal themes and stressors related to the impact of accountability at each level. 
This process is similar to axial coding in grounded theory (Strauss & Corbin, 1990), as it 
involved grouping and comparing the initial codes with each other and merging similar 
concepts together into larger encompassing themes. During this process, all key ideas, 
findings, and interpretations were presented and discussed by EPRRI staff and, at the 
state and district level, with study participants (Miles & Huberman, 1994). Original 
transcripts were used as evidence to support the emerging themes, and at all times the 
researchers searched for examples that would contradict key findings (Yin, 1989). 

II. MARYLAND ASSESSMENT AND ACCOUNTABILITY SYSTEMS 

Each of the four states that participated in this research has a unique 
accountability system to determine the academic achievement of their students. 
Information about each state’s accountability system was retrieved from policy 
documents and other sources, primarily the Internet. Examples of the types of documents 
reviewed include Board of Education policies and minutes, strategic plans, reports from 
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Superintendents and Commissioners, Maryland State Department of Education (MSDE) 
memos, policies relating to standards-based reform, assessments, and accountability, as 
well as press releases and other reports. 

Maryland has been engaged in school reform since the mid-1970s, when concern 
for the quality of public education became widespread in Maryland and in the nation. In 
the early 1970s, it became evident that some high school graduates could not read well 
enough to function in our society, or, if they could read, they were still ill-prepared in 
other ways to take their places as productive adults. The focus on basic skills and 
minimal competencies led to the development of Maryland’s Project Basic beginning in 
1977 ( http ://www.mdk 1 2 . org/mspp/reform/mspap .htmO . The 1980s saw rapid changes 
toward an information- and technology-based global economy and in 1989 the 
Governor’s Commission on School Performance placed a compelling emphasis on 
systemic school reform and education restructuring. Commission recommendations, 
which called for new measures of school accountability for higher academic performance 
by all students, marked the beginning of a new era in Maryland public education. 
Maryland Student Learning Outcomes 

The Maryland School Performance Assessment Program (MSPAP) began in 1990 
in response to the report of the Governor’s Commission on School Performance. The 
program includes a major strategy designed to improve educational opportunity and 
achievement by each student enrolled in Maryland’s public schools. The Commission 
called for a school perfonnance program based on three premises: 

1 . All children can learn. 

2. All children have the right to attend schools in which they can progress and learn. 
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3. All children shall have a real opportunity to learn equally rigorous content. 

In 1990, five Learning Outcome Development Committees, each composed of 
state and local school system content supervisors, developed the Maryland Learning 
Outcomes (MLOs). These outcomes specify what students should know and be able to 
do, by grades 3, 5, and 8, as a result of their educational experiences in Reading, 
Mathematics, Writing/Language Usage, Science, and Social Studies. The Core Learning 
Goals also address the essential skills and knowledge that should be expected of 
Maryland high school students as they prepare for life in the 2 1 st century. They were 
based on national and international studies of student achievement, recommendations of 
national educational reports, and on Maryland's curriculum frameworks. In 2002, a 
voluntary state curriculum (VSC) was established based on the state learning outcomes, 
Maryland content standards, and core learning goals. 

Maryland State Assessments 

Elementary and middle school assessments. In May of 1990, the Maryland State 
Board of Education approved the Maryland Student Learning Outcomes, which set ten- 
year targets for school performance. In addition, an accountability system was developed 
to assess schools’ progress toward achievement of these learning outcomes. This system 
became known as the Maryland School Perfonnance Assessment Program (MSPAP). 
The MSPAP assessments were administered each May to Maryland's 3rd, 5th, and 8th 
graders to test their mastery of the basics and how well they applied knowledge in 
authentic problem-solving situations. The primary purpose of the MSPAP assessments 
was to provide information that could be used to improve instruction in schools. The 
assessments were used to measure the performance of Maryland schools, not of students. 
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Thus, individual student scores were not reported to schools or parents. 

The MSPAP assessments measured how well students solved problems 
cooperatively and individually, how well students applied what they learned to real-world 
problems, and how well students could relate and use knowledge from different subject 
areas. The MSPAP assessments were criterion-referenced performance tasks linked to 
student content standards. Performance tasks were administered in Reading, 

Mathematics, Writing/Language Usage, Science, and Social Studies. Because the 
MSPAP was designed for school accountability purposes and not to measure individual 
student achievement, only a portion of the entire assessment was administered to any 
given student. 

Maryland abandoned the MSPAP after the 2001-02 school year and implemented 
the Maryland School Assessment (MSA) during the 2002-03 school year. Currently, the 
MSA measures student achievement in grades 3-8 Reading and Math and grade 10 
reading: it is based on the Maryland Content Standards. The MSA is also given in 
Geometry after students complete a geometry course, regardless of the students’ grade 
level. Unlike the MSPAP, the MSA meets the federal requirements under NCLB that 
state assessments be administered annually and yield individual student scores. To build 
the MSA, testing experts began with commercial, norm-referenced tests to produce a 
norm-referenced score. To create a criterion-referenced component, test items that 
matched the Maryland Content standards were identified and new items were written to 
ensure coverage of the content standards. The fonnat of MSA is a mixture of multiple- 
choice questions and short-essay questions. 

High school assessments. During the 1980s, Maryland Functional Tests (MFTs) 
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in the four areas were developed and put in place as graduation requirements, with all 
four tests required for graduation beginning with the class of 1989. Maryland 
discontinued the MFTs in 2004 and phased in the High School Assessments (HSA). 

These are end-of-course assessments that measure students’ knowledge in the core 
subject areas of English, Algebra/Data Analysis, Government, and Biology. The 
Geometry HSA fulfills the NCLB requirement for a grade 10-12 Mathematics 
assessment. 

The first administration of the tests was scheduled for 1999 in preparation for 
putting the requirement in place in 2001 for 9th grade students, who would have had to 
pass the new tests by the time they graduated in 2004. However, to give students and 
school systems time to adjust to the new requirements, the state postponed the 
requirement and decided that beginning with the graduating class of 2009 (students 
entering 9 th grade in Fall of 2005) students will be required to earn a satisfactory score on 
all four of the HSAs to earn a Maryland High School Diploma. Students can also earn 
their diploma by earning a minimum score on each test and obtaining a combined score 
that exceeds the total of the four passing scores. This is called the Combined Score 
Option. To use this option, the combined total of a student’s HSA scores must be at least 
1602, with no score below 386 in English, 402 in Algebra, 391 in Biology, and 387 in 
Government. In this way, students may be able to use their higher perfonnance on one 
test to compensate for a lower score on another test. In addition, if a student does not pass 
a test, he or she must receive academic assistance from the local school system and is 
allowed to retake the test as many times as needed to pass it. 

Alternate assessments for students with disabilities . Prior to 2003, students with 
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severe cognitive disabilities who were not able to participate in MSPAP, even with 
accommodations, were permitted to take the state’s alternate assessment, known as the 
Independence Mastery Assessment Program (IMAP). The IMAP assessment tested 
students in their functional life skills in the areas of Academics, Communication/Decision 
Making, Career/Vocational, Community, Recreation/Leisure, and Personal Management. 
The IMAP assessments consisted of an on-demand performance assessment accompanied 
by a portfolio. Student IMAP scores were not counted in the accountability calculations 
prior to the NCLB. Between three and five percent of Maryland’s special education 
students were eligible to take the alternate assessment. Students participated in IMAP in 
the same grades as they would participate in MSPAP (grades 3, 5, and 8) and in grade 11. 
A student for whom the IMAP would be inappropriate because of severe medical 
complications may be excused from the IMAP administration, as determined by a 
student’s IEP team and documented in the IEP. 

No students are exempt from participation in the current Maryland Assessment 
Program (MAP), and all students must participate in either MSA or the Alternate 
Maryland School Assessment (ALT-MSA). When the MSPAP program ended, the IMAP 
was also phased out and replaced by the Alternate Maryland School Assessment (ALT- 
MSA), which is based on the Maryland standards. Students with severe and complex 
cognitive disabilities who are unable to participate in the MSA even with 
accommodations take the ALT-MSA. Each student’s IEP team detennines whether he or 
she takes the ALT-MSA, although the use of proficient scores based on the alternate 
achievement standards cannot exceed 1% of the total tested population in making AYP 
determinations. Students with disabilities are eligible to take the ALT-MSA if they learn 
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extended Maryland Content Standards or access skills in reading and mathematics and 
participate in a Fundamental Life Skills curriculum. 

The ALT-MSA was developed by a team of experts after reviewing the existing 
Maryland Content Standards, as well as examples of access skills used in the alternate 
assessments of other states. When administering the ALT-MSA, a test examiner team 
selects the reading and mathematics content standards or access skills objectives that a 
student is expected to meet with at least 80% accuracy by the beginning of March in the 
following year. A portfolio is assembled that contains artifacts of student work that 
documents student mastery of these standards. The results are reported at three levels of 
proficiency (Basic, Proficient, and Advanced). 

Modified achievement standards and assessments. On April 7, 2005, Secretary 
Margaret Spellings and the U. S. Department of Education announced new flexibility for 
states that would give eligible states an opportunity to develop modified achievement 
standards and assessments based on those modified standards (Federal Register, 2005). 
The state of Maryland has developed procedures, yet to be approved by U.S. Department 
of Education, to take advantage of this additional flexibility. IEP teams determine 
whether students with disabilities will participate in the MSA, Modified MSA (Mod- 
MSA), or Alt-MSA. Students will be assessed using the Mod-MSA if they are learning 
modified academic content standards, require both modifications and accommodations 
during assessments and instruction, and use a modified general curriculum. Students 
eligible to take the Mod-MSA also must have had at least three consecutive years of 
individualized instruction and must not be progressing on grade level. Finally, these 
students must demonstrate that they cannot achieve proficiency in grade-level MSA, even 
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with accommodations. 



Modified and comparable HSAs. The Modified HSA (Mod-HSA) is currently 
being developed in Maryland (MSDE, 2005). Beginning with the 2006-07 school year, 
the Mod-HSA will measure student achievement in algebra/data analysis (Mod- 
algebra/data analysis) and English (Mod-English). Modified assessments in biology and 
government will be implemented in upcoming years. Guidelines for determining 
students’ eligibility to take the Mod-HSA are identical to those used for the Mod-MSA. 
Maryland has also convened a committee to recommend a design for a Comparable HSA 
for students unable to demonstrate their knowledge on the traditional HSA. Both the 
Modified and the Comparable HSA could be used to fulfill the HSA graduation 
requirement. 

Accommodations. Students with disabilities currently have access to certain 
accommodations when taking tests within the Maryland Assessment Program (MAP), 
provided that the accommodations are aligned with and are a part of daily instruction, and 
are justified and documented in the student’s Individualized Education Program (IEP), 
Section 504 Plan, or English Language Learner (ELL) Plan. Since the implementation of 
NCLB, there have been some changes concerning permitted accommodations in 
Maryland. Roughly half of the selected accommodations were the same for 2001, 2003, 
and 2005; however, there have been some notable changes. For example, students using a 
Braille version of the CTBS/5 test in 2001 did not have their scores aggregated, but in 
2003 and 2005 this accommodation may be used without implications for scoring. Also, 
while scores for students using a spell-checker in 200 1 were invalidated for the language 
use portions of the CTBS/5, MSPAP, and HSA, this accommodation was not mentioned 
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in 2003 materials and is currently allowed for students. 

Table 1. Maryland, Selected Accommodations, 2001, 2003, and 2005. 



Accommodation 


2001 


2003 


2005 


Large Print 


A 


A 


A 


Braille 


AI 


A 


A 


Read Aloud Questions 


AI 


AC/AI 


AC/AI 


Magnification Equipment 


N 


N 


N 


Amplification Equipment 


A 


A 


N 


Spell-checker 


AI 


N 


A 


Calculator 


AC 


A 


A 


Scribe 


A 


A 


A 


Write in Test Booklets 


A 


A 


A 


Extended Time 


AI 


A 


A 


With Breaks 


AC 


A 


A 


Individual Administration 


A 


A 


A 


Administration in Student’s Home 


A 


A 


A 



Note: A = Allowed; AI = Allowed with implications for scoring and/or aggregation; 

AC = Allowed in certain circumstances; P = prohibited; N = not addressed. This table 
is based on information from three NCEO reports - Clapper, Morse, Lazarus, Thompson, 
& Thurlow (2005); Lazarus, Thurlow, Lail, Eisenbraun, & Kato (2006); Thurlow, 
Lazarus, Thompson, & Robey (2002). 



As indicated in Table 1, questions could be read to students who needed that 
accommodation when the MSPAP was used for accountability purposes; however, 
students who used the read-aloud accommodation automatically received the lowest 
possible score on the test. In 2002, when test scores were first used for NCLB 
accountability purposes, there was much turmoil because most schools in the state were 
placed into the “Needs Improvement” category as a result of the widespread use of the 
read-aloud accommodation. In 2003, after overwhelming school district and public 
demand, the Maryland accommodations policy was revised. Questions can no longer be 
read aloud on the grade 3 and 4 general reading processes part of the MSA. If the read- 
aloud accommodation is used at other grade levels, the test administration is considered 
to be nonstandard — however, the scores are invalidated only for certain portions of the 
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MSA directly related to general reading processes. 



Maryland Accountability System 

Prior to 2002, each school was given a School Performance Index (SPI), which 
was a composite of each school’s performance on the MSPP standards. The SPI was the 
weighted average of a school’s relative distance from the state’s level of Satisfactory 
performance. SPI calculations included attendance, MSPAP scores, MFT scores, and 
dropout rates for high schools. In order to meet the Satisfactory level of performance on 
MSPAP, 70% of a school’s students had to score at Proficiency Level 3 or above on the 
assessments. To meet the Excellent standard, 70% of students in a school had to score at 
Level 3 and above, with at least 25% of the students scoring at Level 2 or higher. MSPAP 
results were reported through five Proficiency Levels with Level 1 being the lowest level 
of proficiency and Level 5 the highest. On the MFTs, Satisfactory was defined as 90% of 
students passing all tests, while Excellent was 96% of students passing all tests. For 
attendance rate, 94% was Satisfactory and 96% was Excellent, and for dropout rate 3% 
was Satisfactory and 1 .25% was Excellent. Perfonnance standards were established for 
both schools and school systems. Maryland produced School Performance Reports, or 
report cards, which served as the primary accountability tool for MSPAP at the school, 
system, and state level. An SPI index score of 100 indicated that on average a school was 
meeting the state’s performance standards. A School Improvement Team, which is 
present in every public school in the state, was charged with analyzing school and district 
MSPAP data and using it to adjust curriculum and instruction to meet the Maryland 
Learning Outcomes (MLOs). 

Sanctions and awards were determined based on these report cards. Schools that 
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experienced significant progress were given monetary rewards and recognition. MSDE 
conceded in 2001 that the state standard of 70% of students meeting proficiency was 
challenging. At that time, the state had not yet met any of the MSPAP standards. In 1999, 
only 8 of 24 school systems had more than 50% of their students scoring at Level 3 
proficiency or higher. For this reason, focus was placed on schools’ improvement of 
scores, rather than their actual percentage or comparison to other schools. Thus, only 
those schools with large portions of students scoring below Level 3 that were seriously 
below standards and were declining or not making substantial and sustained progress 
were identified by the state as reconstitution-eligible schools. Such schools were obliged 
to submit proposals for how to resolve the problem and, after approval, received 
supplemental funding, technical assistance, and monitoring. They were expected to make 
major changes in staffing and school programs. Reconstitution-eligible schools were 
removed from the list when they showed improvement for three consecutive years and 
met the state average SPI. In certain cases, when schools still did not make sufficient 
progress, the state intervened with state reconstitution. In 2000, the first three schools in 
Maryland were reconstituted, meaning that oversight was shifted from the local school 
system to a third party. 

The cornerstone of Maryland’s new accountability system is Adequate Yearly 
Progress (AYP). AYP replaces the SPI as the method by which Maryland tracks 
academic progress and makes accountability decisions. The norm-referenced score from 
MSA will be used to compare students’ performance to their same-grade peers nationally, 
while the criterion-referenced score will demonstrate how well the students mastered the 
Maryland Content Standards. The state will use only the criterion-referenced score to 
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make accountability decisions for AYP. Maryland is one of the few states that use a 
compensatory strategy to aggregate the scores of each student. Scores for students who 
use a nonstandard accommodation — or who are unable to take a portion of a test — are 
based on the remaining sections of the test. In Maryland, the aggregate score is the total 
score across all items and dimensions being assessed. 



III. MARYLAND STATE AND LOCAL DISTRICT DEMOGRAPHICS 



In the 2000-01 school year, Maryland enrolled over three-quarters of a million 
students and identified almost 30% as economically disadvantaged (Table 1). Just over 
13% of Maryland students received special education services and just under 3% received 
services for English learners. In 2000-01 about 53% of Maryland’s students were White, 
just over 37% were African American, 4.8% were Hispanic, and the remaining were 
Asian, Pacific Islander, American Indian, or Other. The average student-to-teacher ratio 
was 15.3 students to 1 teacher and the average operating expenditure per pupil was 
$7,622 (see Table 2). 



Table 1: State and district select demographics 2000-01 school year 



State/School 

District 


Enroll- 

ment 


% 

FARMS 


% 

Students 

Receiving 

Special 

Education 

Services 

(K-12) 


% 

Students 

Receivin 

g 

English 

learner 

Services 


Ethnic Group 


% 

African 

American 


% 

Hispanic 


% 

White 


% 

Other 


Maryland 


852,929 


30 


13.1 


2.8 


37.1 


4.8 


53.4 


4.7 


Carroll 

County 


27,528 


8.0 


12.2 


0.3 


2.3 


0.8 


95.6 


1.2 


Montgomery 

County 


134,180 


22.7 


14 


7.6 


21.2 


16.2 


49 


13.6 



FARMS=Free and Reduced Meals 
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Table 2: Selected state and school district staff and financial statistics, 2000-01 
school year. 



State/School District 


Average Number of 
Students/ Teacher 


Total Expenditure per pupil 
for general education 


Maryland 


15.3 


$7622 


Carroll County 


17.5 


$6582 


Montgomery County 


15.7 


10,200 



Carroll County Public School System (CCPS) is located near both the Baltimore 
and Washington, DC metropolitan areas. The region can best be characterized as a rural 
area, rapidly becoming suburban. In 2000-01 the student population was just over 27,500 
students. About 8 percent of students were economically disadvantaged in the 2000-01 
school year, much lower than the state overall. About 12.2% of CCPS students received 
special education services and only a tiny percentage received services for English 
language learners. Over 95% of students were White and 2.3% were African American. 
The number of students per teacher (17.5) was slightly higher than the state average and 
the operating expenditure per pupil ($6,582) was lower than the state average. 

Montgomery County Public School System (MCPS) is located in the Washington, 
DC suburbs and is the 17th largest in the United States. In 2000-01, the student 
population was just over 134,000. Almost 23% of the students in MCPS were 
economically disadvantaged, a lower percentage than the state as a whole. Almost 14% 
of students received special education services and almost 8% received ELL services. 
MCPS had a diverse student body of just under 50% White, over 21% African American, 
16% Hispanic and almost 14% Asian or Pacific Islander. The number of students per 
teacher was 15.7 to 1, which was close to the state average, with an operating expenditure 
per pupil of $10,200, considerably higher than the average for the state as a whole. 
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IV. MARYLAND EMBEDDED STATE AND DISTRICT FINDINGS 



Maryland has been engaged in school reform since the mid-1970s, when concern 
for the quality of public education became widespread in Maryland and in the nation. In 
the early 1970s, it became evident that some high school graduates could not read well 
enough to function in our society, or, if they could read, were nonetheless ill-prepared in 
other ways to take their places as productive adults. Maryland embarked on an innovative 
approach to education reform in the 1990s and, unlike many other states, maintained a 
course faithful to its original design and timeline. The MSPAP was designed to guide the 
direction of education reform for the succeeding ten years and provide infonnation on the 
progress made by districts and schools toward achievement of student learning goals. 

The data from Maryland indicate that at the state and district levels there was 
considerable support for accountability reform, using data to guide improvement, and 
closing the achievement gap between minority and non-minority students. Consideration 
of the progress of students with disabilities was not, however, an integral component of 
the improvement process, although there was some evidence that the situation was 
beginning to change — especially in relation to high stakes assessment of students. 
Findings are arranged into three broad topical areas: (a) upcoming changes to the 
accountability system, (b) building instructional capacity, and (c) reform in special 
education. Within each theme are related subthemes that vary according to the time 
period and level from which data were collected. 

State level interviews took place in September 2001 and district interviews took 
place in June 2002. EPRRI staff interviewed seven individuals at the state level and three 
at the district level. In each broad area, the data will generally be presented at the state 
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level and then the district level. The significant conceptual themes that emerged will be 
illustrated by a small number of exemplar quotes. Each quote was chosen to be 
representative of many such instances, or was chosen because it was an extreme instance 
or the sole instance, or represented two contrasting properties of the conceptual theme 
under discussion. 

Changes to the Accountability System 

Although the accountability system in Maryland can be characterized as relatively 
stable compared to other states, when we visited the state in October 2000 informants 
discussed several planned changes designed to increase the rate of progress toward the 
state’s student learning goals. In addition, some informants discussed the upcoming 
reauthorization of the ESEA and speculated on its potential impact on the state’s current 
system. In this section, we will address the following themes: (a) changes designed to 
improve alignment between the Maryland learning standards and classroom instruction, 
(b) the issue of a state curriculum, (c) introduction of the High School Assessment 
(HSA), and (d) the state system after the reauthorization of ESEA. 

Improving alignment. Although schools and school systems were making some 
progress toward meeting state learning standards, the rate of progress was slow and had 
stagnated to some extent, especially at the middle school level in Mathematics. 
According to several informants, part of the problem was the lack of alignment at the 
local level between curriculum and instruction and the state learning standards. The state 
had earlier developed content standards in grade bands, leaving it up to the local systems 
to decide the scope and sequence of learning across each content area: “We have content 
standards at K-3, 4-5, 6-8, and 9-12, four grade bands. They are about two years old, 
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most of the schools systems have been working with them for a while; they represent 
Math, Science, Social Studies and ELA.” However, as one informant pointed out: “They 
[school districts] made an effort, but some counties have done it better than others.” To 
help districts, the state was developing grade by grade standards that were back-mapped 
from grade 8 down to pre-kindergarten: “We worked with the content areas to backward 
map, basically from high school to 8 th grade, to 7 th , 6 th , all the way down to pre-K. Now 
those are not scope and sequence over the course of the school year.” 

In developing the MSPAP, the state intended that instruction in classrooms should 
be changed as teachers modeled how to apply skills to real-life situations. The MSPAP 
calls for students to apply skills to solve real-life problems, to work in groups as well as 
individually, and to integrate knowledge from different subjects. Thus the testing 
program was intended to mirror good instruction. However, as the following quotation 
illustrates, state level informants reported that some local school systems did not initially 
understand what the test was about and developed their own “practice” tests. These tests 
were not reflective of the MSPAP and did not lead to improved performance as the 
districts hoped: 

Initially, there was a big lack of understanding of what the test was all about so 
there was big push among local school systems to create their own practice tests. 
However, because they really, truly didn’t understand what was being measured 
by the state test, their format looked the same but the actual correlation between 
what was being done at the school level and the local level was no match for what 
was being done at the state level. 

One of our study districts confirmed that it had developed its own assessments 
patterned on the MSPAP and the HSA: “We’ve developed a ha nk of state assessment-like 
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items we’ve patterned after the format that we do administer locally.” This informant 

explained that the district held “mock” assessments just before the actual MSPAP to help 

identify areas of weakness in time for remediation: 

For example, English I, which is a high school assessment, we give a mock high 
school assessment not used as part of the grading but it’s given like a month-and- 
a-half before the high school assessment which will give the teacher 
information — okay maybe the students need more information about writing 
rather than reading or whatever, but it’s patterned after the high school 
assessments. We do the same thing with middle school and elementary school. 

To help districts and teachers better understand the link between instruction and 

the MSPAP, the state developed instructional strategies. One informant stated: “I think 

the problem is in the actual instruction and what goes on behind closed doors in a 

classroom.” Embedded in the new grade level standards were instructional “seeds” that 

provided teachers with models of how to teach a particular skill: 

We do have some manuals that we’re developing that we’re calling instructional 
seeds that are ideas for instruction. How to take those standards and to work them 
into the structure and delivery of a lesson and how you would assess each of the 
indicators that measured standards. 

However, these instructional “seeds” did not contain strategies that were specific 
to helping teachers work with students in special education, although they did have 
support for English language learners and other at-risk groups: “I believe there’s some 
specific suggestions for ESL students. I don’t know that there’s specific suggestions for 
special education. I think there may be specific suggestions for students at risk, but not 
for students with disabilities, per se.” 

When we visited the districts in June 2002, infonnants also addressed the issue of 



27 




alignment of the local curriculum with the state’s learning standards. As the following 
comment illustrates, school district content specialists had been involved in the decision- 
making process at the state level and attributed their performance on state assessments to 
alignment: 

I think we owe a lot of the results to the fact that our content area supervisors 
were directly involved on the state committees that helped develop these 
standards, the core learning goals, and the assessments. So we really had a direct 
link into the process. So those folks would come back to the county and then 
certainly play a key role in aligning our curriculum with what the state was 
expecting. 

Informants in one of our districts told us that special education teachers were 
involved in developing the local curriculum. The role of the special education teacher 
was to provide classroom teachers with classroom-based strategies that they could use to 
help support students with disabilities: 

There is a special education teacher involved on every curriculum writing team. 
She helps tailor lessons and gives us suggestions on how to modify lessons to help 
students with special needs. We have special education documents on every 
curriculum guide that will help the classroom teacher, because we want the 
classroom teacher to see that they are the first intervention, that if you have a 
student with special needs in your class it’s not someone else’s problem, you deal 
with it. 

In addition, one of our districts developed three different curriculum guides for 
students of varying abilities. Each student was expected to reach the same learning 
outcomes, but teachers were provided with different pathways to get them there. As the 
following quotation illustrates, the guides varied by instructional strategies and length of 
time taken to cover the content rather than having different content: 
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I don’t care if you have an IQ of 120 or 80, you have the same outcomes that you 
need to master. What we have is three different curriculum guides, same 
outcomes but just a different way of approaching those outcomes. We have an 
honors track, the academic track and then we have the fundamental track, which 
is more time. We hope that time and instructional strategies are the variable rather 
than the content. 

Development of a state curriculum. According to our state level informants, some 

people thought that the new grade-by-grade guides were very much like a state 

curriculum. One person commented: “Some people would say they are very close to a 

state curriculum.” Indeed, one infonnant stated that people had argued that a state 

curriculum was established in the early 1990s when the learning outcomes were 

developed: “Philosophically, some folks might say that we’ve had a state curriculum 

since 1990 when the first Maryland learning outcomes were produced.” However, several 

state informants told us that the state’s advisory board was likely to recommend that the 

state develop a voluntary state curriculum: “The current requirements in bylaw is that 

school systems have the curricular frameworks in place and reflected. What they are 

probably going to say is that we need a state curriculum.” 

State informants also predicted that a voluntary state curriculum would be greeted 

by some districts with delight and with others by horror: 

The Visionary Panel I’m pretty sure is going to recommend a voluntary state 
curriculum and I can identify school systems that are large, medium and small 
who want it and others who say this is the craziest thing, why are you doing this? 

Another infonnant confirmed that the local counties were split on whether the 
state should develop a voluntary curriculum with some completely against the idea and 
others wanting the state to step in: 
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Some are having fits but others want the state to be more for them in that regard 
because the state, right now, does the four learning goals as Maryland Learning 
Outcomes, but it hasn’t gone back to the curriculum level and that’s what they 
want. Everything except for like selection of textbooks and instructional 
materials. 

At the district level, informants commented that the state already had a curriculum 

defined by grade span, but that a more detailed guide couldn’t do any harm: 

I think we have a statewide curriculum. In all the content areas we had 
assessments, we have the content standards, and they’re really grade by grade 
strands already, they’re clusters. So we weren’t asking for a state curriculum, but I 
don’t think it will hurt. 

Introduction of the High School Assessment (HSA). As part of the state’s 
commitment to improving the quality of its high school graduates, Maryland planned to 
phase out the Maryland Functional Tests and introduce the more challenging HSAs as 
one of the requirements for graduation with a high school diploma. In the design of the 
HSA and the overall assessment system, the state clearly considered the needs of students 
with disabilities: 

If the kid has an IEP we honor that IEP on the assessment. So if the kid’s IEP says 
they need a calculator then on the assessment we would say you can use a 
calculator. We honor that whether it’s HSA, functionals, MSPAP, CTBS. 

As the following statement indicates, Maryland sidestepped the controversy 
around certain accommodations faced by several other states. The read-aloud 
accommodation on the reading portion of similar exit level exams proved problematic in 
other states, but Maryland was considering the adoption of the Kurzweil 3000 screen 
reader. As the following quotation indicates, MSDE staff had met frequently to discuss 
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the construct of reading and had determined that for the HSA the construct was 

comprehension and not the act of decoding: 

We’re looking at right now, the use of the Kurzweil System for anybody that 
needs reading as an accommodation, but that has to be what’s used. We had a lot 
of departmental discussions on this and the whole purpose of reading in the ninth 
grade is not decoding, it’s understanding. So the legislature agreed on this so that 
kids can take the test, get a score, and graduate. 

Although the state avoided controversy over accommodations, informants did 

address other aspects of the HSA that they were concerned about. As the following 

comment illustrates, there was a group of student with disabilities who could meet all the 

other requirements, but who would not be able to pass the more challenging HSA: 

There is a relatively substantial group of students who will be pursuing a high 
school diploma and therefore be enrolling in courses, and who are able to 
complete the course content and get the credit, but not be able to pass some of the 
assessments. So we’re looking at what are the reasonable options for those kids 
and there are several. 

This infonnant went on to point out that these students could get a program 
completion certificate, similar to the ones students with more severe disabilities who 
were not following the state’s core learning outcomes received. Another option would be 
an IEP diploma certified by the IEP team that the student had met all the requirements 
other than the HSA: 

They could obviously just get a program completion certificate. They could get an 
IEP diploma and the IEP team certified that they have met all of the other 
requirements for a high school diploma, but they cannot possibly pass all the 
assessments or they might have two of the assessments, passed all the credit 
requirements, something like that. 
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Finally, this participant added that they were very concerned about this group, as 



the lack of a high school diploma could have serious consequences for the students 
affected. As this comment illustrates, the students who were likely to be most affected 
were the ones who had the potential to succeed in postsecondary education if they had 
sufficient supports. However, to enter many post-secondary education systems a student 
needed a high school diploma: 

So that’s the “sped” group that we’re very concerned about. This group could 
contain a lot of kids, in all probability, who are learning disabled or emotionally 
disturbed. And those students have the greatest likelihood, with support in 
postsecondary programs, to be successful. So without a diploma, the impact could 
be extraordinary for them. So we’re looking at what would be the kind of exit 
document that would be appropriate for them. 

Another infonnant addressed the issue of local alignment and consistency with 

the core learning goals and the implications for HSA, because of the high stakes for 

students. As the following comment indicates, the opportunity to learn had to be assured 

for all students, but students with disabilities were at risk depending on their placement: 

The Department of Instruction is doing local audits across course content and 
curriculum to make sure the core learning goals are, in fact, being addressed. For 
instance, if you’re enrolled in Algebra that you’re covering those core learning 
goals. We have to look at placement of students with disabilities, if they’re not in 
courses that have the content, then they could never pass the test. 

As the following quotation indicates, introducing more rigorous assessments at 
the high school level had implications throughout the system and especially on teacher 
certification. If graduation became dependent on passing the HSA, then students had to 
have certified teachers or the system would not work and parents would react: 
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The high school assessment brings on a couple of dilemmas. One of the reasons 
why we’re moving aggressively to get our certification process modernized is 
because once high school assessments kick in for real it will beg the question that 
parents will naturally ask — was my son or daughter’s teacher certified and what 
in? 



The state system after the reauthorization of ESEA. When we conducted the state 

level interviews in September 2001, there was considerable speculation regarding the 

impact of the forthcoming reauthorization of ESEA. One informant commented: “I think 

the other piece — the wild card right now — is the reauthorized Elementary and Secondary 

Education Act, and that’s liable to change everything.” Several informants speculated 

that MSPAP would change, as the assessment did not generate individual student level 

scores: “I think the Visionary Panel is looking at MSPAP being inadequate — it informs 

school programs but there should be child level reporting and I don’t know where the 

state is going, but I think the Feds were fooling around that.” One informant added that if 

the National Assessment of Educational Progress (NAEP) became the nation’s 

assessment, then states would change their state assessments to align with it if the federal 

government were serious about accountability: 

If NAEP becomes the national assessment and you have 80% of your kids passing 
reading on their state assessment and you’ve got 20% at proficient on NAEP, I’ll 
bet your assessment will change. It depends on whether the Feds decide to tie up 
funding. That’s another whole issue. Clearly it looks like NAEP will become a de 
facto national test. The question would be, is there any accountability or is it just 
going to be a press release? 

When we conducted the district interviews in June 2002, the situation regarding 
the MSPAP had become even more complicated. Bowing to pressure from some districts 
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and in order to comply with NCLB, Maryland announced that the state assessment would 
change by the next administration. However, several district level informants expressed 
frustration at the lack of information on the new assessment and the changes this would 
require at the curriculum level: 

What’s most disturbing to us is really the lack of information and the lack of 
guidance. If this is going to be in place next school year we need to know because 
all of our curriculum writing happens in the summer, so if they come out next 
September with the new assessment, I mean, it will be too late for us to do 
anything for that school year. So we would be more reactive instead of proactive, 
which I don’t think is a good thing. Our schools are basically on hold until the 
state comes out with what they’re doing. 

Another district informant explained that the choice of assessment could have a 

huge impact on the district, as it may require substantial reworking of the curriculum and 

substantial professional development for teachers: 

One thing I’m concerned about is we have our content standards and our MLOs 
and whatever, but if they choose a national test or some other assessment that will 
replace MSPAP, it’s not going to be a clean fit with the content standards because 
they’re developed locally. I think what they’re more apt to do is choose a national 
assessment that’s already developed. So that will cause us to have to revise our 
curriculum because obviously we’re going to have to align our curriculum 
according to the assessment program. So it will cause us some curriculum 
revisions for sure and staff development needs to help teachers. 

In this section, we have discussed the importance of alignment of the local district 
curriculum with the state’s learning outcomes for students and the state’s assessment. 
Although the system had remained stable for some time, there were indications that some 
districts had yet to align their local curriculum with the MLOs. The likelihood of a state 
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voluntary curriculum to assist districts further was voiced by most state level informants. 
At the district level, our informants believed that they had aligned their curriculum 
closely with the MLOs. The introduction of the HSA and its implications for students 
with disabilities was also addressed. Although the state had not fallen foul of the 
accommodations issue as other states had, the issue of the high-stakes nature of the 
assessment for student with disabilities was seen as problematic. Finally, the 
reauthorization of ESEA was described as a great unknown by the state, and at the district 
level informants expressed concern over the impact of the state’s decision to change the 
assessment for the upcoming school year. 

In the next section, we present findings on the extent to which the state and the 
local districts have the capacity to implement the state’s ambitious standards-based 
reform initiative. Informants will talk about overall capacity to implement reforms as 
well as their capacity to implement reform for students with disabilities in particular. 
Capacity to Support Reform 

The success of standards-based reform depends heavily on the ability of all levels 
of the public education system to respond in appropriate ways to the information 
provided by accountability systems. In this section we address the issue of capacity 
across three related domains: (a) the state and the districts’ capacity to direct resources to 
support reform, (b) educators’ knowledge and skills, and (c) campus principals’ capacity 
to be instructional leaders. 

State and District Capacity to Support Reform 

Financial Resource Limitations. The issue of how to best allocate valuable and 
scarce resources to support reform was discussed at both the state and district level. At 
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the time of our state and district interviews, both entities were experiencing significant 

budget shortfalls, which impeded their ability to support the state’s reform agenda and 

particularly to respond to the changes in the assessment system, and the implementation 

of the HSA requirement that additional instructional assistance must be provided to 

students who do not achieve the passing score on the HSA before they can retake the test. 

One state level participant stated bluntly: “We don’t have money.” Another informant 

explained that MSDE had frozen their budget, which included money for the MSPAP and 

the HSA: “You know we had to freeze almost a third of our headquarters’ budget which 

includes much of the money for high school and MSPAP.” 

This same participant explained MSDE had a large aid budget but a small 

headquarters budget, out of which came financial resources to support the HSA and 

MSPAP. Some of the money was to provide remedial help for students who failed one or 

more HSAs. As the following comment indicates, if the money to support the HSAs was 

not made available, then the state could not make them a requirement for graduation: 

You know we have a huge aid budget, then we have a tiny headquarters’ budget. 
We’re not allowed to touch the aid budget so all of the money that our agency has 
to come up with is our headquarters’ budget. We are hoping that they will release 
a small portion when they get the next budget working, so we can move forward 
with some of the plans for HSA. Otherwise we’ll have to push the HSA 
requirement off because we have no money to do it. You know, you can’t just 
pretend you’re doing something, you actually have to do it. 

A similar picture emerged at the district level as informants explained that the 
budget shortage at the state was mirrored in the districts. As this informant explained, the 
district had to wait to see what the new state assessment would be before they changed 
their curriculum, and they didn’t want to guess and be wrong: 
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Resources are, as you know, tight around the state and quite limited in [our] 
county and we can’t go out and fix our curriculum on what we think is going to 
happen, we’re waiting to see what they actually come out with. 

Human resources limitations. Like many states, Maryland was experiencing a 

shortage of teachers and administrators. As the following comment illustrates, as 

experienced administrators and teachers left the local systems they have been replaced by 

individuals who, for various reasons, do not understand or internalize the state’s 

education reform agenda and need to be trained: 

What is happening as well at local school system levels with maturing of an 
administrative and a teaching population, is that a lot of people that we have first 
communicated the message to have either retired or moved on and the new people 
are coming sometimes with their own prejudices, biases, lack of information and 
so we have to keep repeating the message. 

Informants at the state level indicated that a significant barrier to improving 

failing schools was the inability of some districts to move successful educators and 

principals into struggling schools. One state participant commented: 

Our lowest performing schools in the state are typically ones where we have the 
least experienced teachers and the highest turnover of teachers. As soon as you 
get good in those schools then you are going to be hired to teach in a more 
desirable school. 

However, the state’s ability to help was limited: 

Our role at that level is very limited. We’re dealing with it on two levels; one is 
how do you get the best teachers into those schools, and sometimes finances is not 
enough, it’s not the only thing, versus just trying to get enough people to apply. 
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As the following statement indicates, some school systems did not fully 

understand the importance of teacher quality: 

I can still take you to school systems where one of the assistant superintendents 
said, “Gee, you know, I need you to come out and help me figure out what’s gone 
wrong with this school, we can’t seem to do better.” I said to him, “Are your best 
teachers at that school?,” and he said “Well, of course not.” So I said “Well, what 
do you want me to do?” Now that’s a simplistic way of looking at it. It starts with 
your best teachers and your best principals. Now how do you get them there? 

However, districts that recognized the problem and attempted to address it also 

faced significant barriers. As the following comment from the state level pointed out, 

districts were unable to control where teachers taught because of the underlying issues of 

seniority and mobility options within the district. This informant saw this as a major 

factor in how schools began to fail in the first place: 

Districts are the ones who have to worry, they’re the ones that employ teachers, 
they’re the ones that deploy them as well. And quite frankly, there are districts 
that do a terrible job of deploying their teachers in places where they need them. 
Quite frankly, the way it’s set up is that teachers really decide where they want to 
go. Either by seniority or by the mobility options in districts. I mean, what we’ve 
found, and it’s a no-brainer, but in the reconstitution-eligible schools, basically 
they get that way, or the staffing gets that way, because many of the teachers that 
have seniority move on up and never look back. 

Districts faced resistance in other areas, too. As the following quotation 
illustrates, teacher unions objected to district incentives designed to get teachers to teach 
in failing schools: 

There have been issues around a school system that wants to provide incentive 
pay to work in failing schools, and the union said you can’t do that for some 
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teachers, you’ve got to do that for all teachers. So you run into negotiation issues, 
you run into how to foresee a funding issue, and those are not problems that I 
think school systems have figured out how to solve. 

One of our state level informants described several strategies that a local school 

district had implemented to encourage successful teachers to stay in failing schools: 

Places like [school] has significantly increased first year teacher salary, they’ve 
created eleven month contracts instead of a ten month contract, which is a big 
difference, you don’t have to go hunting for work. And you get paid a high 
amount connected to professional activities. For example, if you commit to 
staying at a poor school they’ll partially support a master’s program and get you 
those extra credits, moving up the pay scale quicker. 

Another infonnant at the state level confirmed that struggling schools had the 
least experienced teachers: “What we’ve seen in our low-performing schools is that they 
far exceed the average of provisionally certified teachers. So what we see is that the least 
experienced teachers have been placed in the most challenging of circumstances.” 
Another infonnant stated bluntly: 

There are middle schools in [name removed] and [name removed] where kids 
probably had a substitute teacher every year, every day of every year in the 
middle school. And a substitute teacher may not even be certified, so the 
likelihood of those kids succeeding is really low. 

Another state informant went on to describe a state level initiative to encourage 

master teachers to work in failing schools: 

Now what we’ve tried to do in the state of Maryland is provide, through our 
Teacher Quality Incentive Act, funds to work in failing schools. It’s a $2000 
stipend for teachers who hold our advanced professional certificate, which is our 
highest certificate, who will work in those schools. It’s on an annual basis as long 
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as the school is a reconstitution-eligible school, a reconstituted school or a 
challenge school. 

Improving Educator Knowledge and Skills 

At the state level, it came as little surprise that the issue of educator capacity was 
very much on the minds of all participants. Responses at the state level reveal little that is 
new on this issue, but rather underscore the pivotal importance of teacher quality to 
student learning: 

The first thing is we’ve got to have the best teachers that we possibly can. To 
move those kids around, the first critical thing is the quality of the teacher. So I 
think that the first thing is stabilizing and upgrading the quality of the teaching. If 
you don’t have a good teacher there’s no curriculum that’s going to meet the 
needs of all those kids. The teacher is the person who sits down with that group of 
kids, the teacher needs to know these are the skill sets and then they need tools to 
help the students get there. 

The district level responses, while not denying that teacher capacity was an area 

of concern at the state level, did not reflect as high a level of immediate concern. In this 

section we will look at three issues: improving the capacity of new teachers, improving 

the quality of incumbent teachers, and developing principals as instructional leaders. 

Preparation of new teachers. The issue of new teacher quality and quantity 

proved to be a very complex problem. As the following remark illustrates, Maryland, 

along with most other states, was trying to improve the quality of new teachers coming 

into the profession by increasing teacher education standards and requirements at the 

precise time that fewer people were willing to enter the profession: 

Well, we’re facing a demographic problem and we’re trying to raise the standards 
for both the content knowledge and the pedagogic knowledge of our new 
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teachers. In addition, we’re trying to do that in competition with the other states. 
At the same time, the number of people who want to be teachers is not probably 
growing or at least not fast enough to solve our teaching problems. That’s a 
complex set of issues. 

Maryland embarked on a redesign of its teacher education system in 1995 as part 
of its overall reform agenda. However, as the following comment indicates, the state 
legislature did not commit funds to support the initiative, which then had to be funded 
through grants obtained by MSDE: 

In Maryland there has been an effort in the last 10 years to increase the 
preparation of our teacher candidates. On the bad side, although there was all 
kinds of recommendations to fund teacher preparation and to really put some 
money behind preparing teachers the right way, there has been no state 
commitment of funds in that regard. So what we here at the Maryland State 
Department of Education and the Maryland Higher Education Commission have 
done is sought funding sources, funding streams through grants. 

The Maryland initiative, known as the redesign of teacher education, had several 

key components including improved content area preparation especially in Mathematics 

and Science, a longer clinical experience, and improved pedagogical knowledge. The 

state made sweeping changes in student teaching requirements in particular: 

The redesign called for year-long internships for teacher candidates, moving away 
from the semester student teaching experience. The clinical preparation should 
occur in professional development schools and in a real setting that allows an 
individual who is pursuing the teaching profession to get really immersed into the 
day-to-day operations of the school. 

As the following remark indicates, the state intended that higher education faculty 
should play an important role in improving beginning teacher quality. The year-long 
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clinical experience was designed to encourage this involvement and be mutually 
beneficial: 



This theoretically allows for higher education faculty to work closely with K-12 
faculty. One of the theoretical underpinnings of this is that teachers in K-12 
schools would benefit from working with higher education faculty who are on the 
leading edge, the cutting edge of teacher preparation. Also higher ed faculty 
would understand the day-to-day operations of schools and can tailor and re- 
engineer their teacher preparation programs to take into account what teachers are 
really being asked to do. 



Another infonnant reiterated the state’s desire to involve teacher preparation 
programs and higher education faculty in the K-12 initiative. However, the following 
response makes clear that the state intended to be in control of what was taught in the 
future: 

There’s always that concern that new teachers that are coming out haven’t a clue. 
They have no clue as to what is expected of them by their employers. We want to 
have a conversation with higher education about the importance of them ensuring 
that teacher candidates understand that they have a role in helping minimize or 
eliminate the student achievement gap. Teacher prep is a very important aspect of 
higher education’s contribution to the quality of our K-12 public schools. So they 
need to be knowledgeable about where those shortcomings are. 



Another state level informant was more critical of the teacher preparation 
programs, commenting: “I’m not convinced that certification programs, in general, 
produce great teachers.” However, this informant was supportive of the Professional 
Development Schools (PDS), pointing out that earlier and longer field experience 
allowed prospective teachers to determine if they really wanted to enter the profession: 
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The PDS schools I think are a great thing, and getting the potential teachers into 
teaching situations early is a wonderful trend. I think all those things are great. If 
you are putting teachers in places where they’re going to have success this gets 
them committed. In the old program I knew students who suddenly as seniors 
when they did their field experiences discovered that “I hate it. I’m not only not 
good at it, but I hate it. I don’t feel safe.” 

Another infonnant reported that newly qualified teachers who had been placed in 
PDS schools were much sought after by the districts because they had more maturity and 
experience: 

Districts tell me they hire every one of the PDS folks. One superintendent told 
me: “They spend a year in our district full-time in a school and we hire as many 
of them as we can because when we get them they have the maturity of a second 
year teacher, they know our system, they know what we expect,” and they go 
after them. 

The state also intended that the redesign be part of the overall reform of education 
in Maryland and at the federal level: “We want a strong connection between teacher 
preparation programs and K-12 priorities in terms of what we’re asking, what the state 
and federal government are asking of teachers and of students so that the teacher prep 
programs wouldn’t be sort of working in isolation.” 

One of the most important goals of the reform in Maryland was to close the 
achievement gap between minority and non-minority students: “There is another priority 
that has been the state board’s priority this year and last year and that will be in years to 
come, and that is eliminating the academic achievement gap.” As part of this the teacher 
redesign required that teacher candidates be skilled in teaching diverse learners: “A third 
component is exposing teacher candidates to a diverse student population. Diversity is 
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broad in the sense that it does include special education. But diversity includes students 

of varying socioeconomic strata and ethnicity.” 

The state also changed the teacher certification test to the Praxis I and II. As the 

following comment illustrates, the new tests allowed the Board of Education to address 

pedagogy as well as content area expertise: “In 1998 Praxis I and II were brought on 

board and it gave the Board the ability to test pedagogy across the content areas.” 

However, several informants reported that the state was able to influence higher 

education and hold it accountable for the quality of new teachers because of the 

reauthorization of the Higher Education Act, or Title II, which requires teacher 

preparation programs to report pass rates on the teacher certification tests and enables the 

state to remove accreditation status for programs that do not improve their pass rates: 

Title II is probably the biggest effort thus far to put some accountability on 
teacher prep programs. Title II requires for the first time teacher prep programs to 
report the pass rates of their program completers on the teacher certification tests. 
The stick is that campuses will have to report in all of their publications if they are 
a low-performing campus. Now, if it is a continuing pattern, the states can remove 
their program approval status, so the students graduating from the campus 
couldn’t get certified by the state and if they fail to comply they can be fined by 
the federal government. So this is a big stick. 

The state was in the process of addressing special education teacher preparation 
programs and in particular, the need for special education teachers to have more content 
area knowledge: “We have a generic special ed endorsement, it’s broken up by grade 
level. . ..But right now there’s no content in that.” As the following response indicates, the 
expectations for what special education teachers needed to do in a school had changed 
and they found themselves in content area classes: 
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Currently, special educators have a lack of content knowledge and in the real 
world environment they are being asked to work with students at a variety of 
different content levels. At the elementary school they’re in classes in language 
arts, math, science and social studies, so it’s trying to come up with a state 
strategy on addressing that problem. A task force is looking at that right now. 

When we visited the districts, the issue of teacher shortages did not emerge as a 

concern. However, in one of our districts, informants reported that it could become a 

problem as many teachers in the district were getting close to retiring. As this comment 

indicates, the teaching force was pretty stable: “People usually stay in [this] county. 

However, unfortunately we now have a lot of our veteran teaching staff who are that 

retirement bubble right now and we will be bringing in a lot more new teachers.” 

Improving the skills of incumbent teachers. Informants at the state level regarded 

this as a very important issue, especially given the fact that many teachers who were new 

to Maryland were not new to the profession: 

Maryland imports a lot of teachers, so that’s why you can’t just tackle it on the 
front end. As teachers come in either with experience from other states or from 
other states with no experience, there’s an in-service or professional development 
component here that needs to be addressed and picked up, but attention needs to 
be focused on the incumbent teacher. 

According to several informants, the state required every county and every 
teacher to have professional development plans in place and approved as a requirement 
for renewal of his or her teaching certificate: “Every school teacher is supposed to have 
professional development plans approved for you as a condition of renewing your 
certificate.” However, there is evidence that special education was not included: 

It’s amazing how many systems do not have staff development plans for special 
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education. I would say the majority do not. Now they may have one for their 
county for general education, but there’s nothing for special ed and so it’s kind of, 
you know, it’s a total blank. But I’m looking to see: “Does it match with what 
your weak areas are, etcetera.” So it may mean that we need to do a lot of work in 
training around that issue. How do you do a long-range special education staff 
development plan? What do you base it on? 

Several infonnants at the state level reported that middle school was an area of 
concern, particularly in mathematics, as the content knowledge required to teach the 
MLO was very rigorous. As this informant explains, well over half of the mathematics 
teachers in middle schools were actually elementary certified and did not have the 
content area skills they needed: 

In Maryland probably 65% of the middle school math teachers are elementary 
certified, so I would argue that if you’re elementary certified that almost 
guarantees that the last math you took was in middle school. You probably took 
the low level high school math and struggled through an Algebra, Geometry and 
maybe I’ll take Algebra II. Then you went to college and you got the same math 
repeated. Now you come to Maryland and I’m going to ask you to do box and 
whisker plots and maybe linear equations, and do it on a graphing calculator. 

Other infonnants at the state level pointed to problems with reading instruction in 
some districts. As the following quotation illustrates, some teachers found it difficult to 
strike a balance between direct instruction of reading skills and a whole language 
approach: 

What I found were teachers that didn’t know what they were doing. Drill and kill. 
There was no translation of why they were doing a skill into do it so you enjoy 
reading. These kids were not deriving pleasure at all from reading and so 
therefore anything to do with reading was sort of like an ouch and my guess is 
that fueled a lot of the whole language surge here. Initially the teachers thought 
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they could jettison teaching of skills to get to the more pleasurable aspect. But 
they didn’t bring in enough of the skills for the kids to build a foundation and 
become independent without the teacher feeding them new words or helping them 
through difficult syntax, or passages that were more abstract. 

A different informant expressed a similar view, suggesting that the problem was 
an instructional one that could be helped by research-based professional development in 
reading: 

I think really we have an instructional issue, I think teachers probably through 
staff development need a lot of mentoring with someone who really knows how to 
manage various reading levels within a classroom. I don’t think the support is 
there in many instances for the teachers who do not know what to do with a 
problem reader. 

This infonnant went on to explain that middle school teachers in particular needed 

to be better prepared to help struggling readers. However, as this comment illustrates, 

teachers at the middle school level expected students to come in knowing how to read 

and did not have the skills to assist them in reading in the content areas: 

I would say that most teachers at the middle school level operate from the 
perspective that these kids should come in knowing how to read and if they don’t 
just give them more work or make them stay longer. It isn’t anything that’s 
helping them. It’s because there was no background in reading required until very 
recently. Even if the teachers had had a little bit of reading instruction, the 
pressure of reading through the content was much stronger than for helping the 
child read independently. 

Because closing the achievement gap was a priority for the state Board of 
Education, developing the capacity of districts and teachers to use data in making 
instructional decisions was as a priority at both the state and district level. However, 
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several informants at the state level reported that the state did not use data to address the 

gap between regular education students and special education students: 

The State is doing a lot of looking at gaps, but it’s between race, you know, 
ethnicity. It’s not looking at kids with disabilities as a group, what do we need to 
do with them as a group is the question. 

As the following remark illustrates, some districts were beginning to look more 

closely at what they may mean in terms of special education performance and 

recognizing the importance of access to the curriculum: 

I’m having them look at their data, and, of course, the gap between the standard 
and where they are. Now, one of the things that’s coming out of that, though, is 
they’re beginning to say: “Oh, wait a minute. What are some of the things that 
may be driving these lower test scores. What kind of a program are kids getting?” 
Now I have some districts — the light bulb has gone on and they said: Well, we 
ought to be seeing if our IEP goals and objectives are aligned with curriculum. 

We ought to be looking at access to the general ed curriculum. 

When we visited the districts in June 2002, we found considerable support for 

accountability reform and also recognition of achievement gaps between certain 

populations. The following quotation is illustrative of the general feeling in the district 

that education systems should be held accountable for student learning and that teachers 

should be evaluated on how well their student subpopulations were perfonning: 

I’ve always said that in teaching we are really not evaluated or held accountable 
for the only thing that we’re paid to do, which is to ensure student learning. 
Teachers are evaluated with absolutely no consideration of how well their kids are 
doing, which really doesn’t make sense. School systems look at are the kids safe, 
are they happy, are they all promoted. But until we actually get down and take a 
look at the data, and not just the holistic data but disaggregated data because we 
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have some issues with the subpopulations in our school system, and until you do 
that you really can’t measure the impact that you have on students. I think it’s a 
great move. 

However, this informant went on to say that this type of analysis was only taking 
place in his district at the central office level and had yet to be established in schools and 
classrooms: 

We’ve had data centrally. We’ve never given data out before to the classroom 
level, and I think data will have the biggest impact when it hits the classroom. 
That’s our goal, our goal is to get it down to the teacher. Centrally we’ve used 
data to revise curriculum, for example. But we haven’t used data anywhere near 
its potential yet. 

As this informant pointed out, the district did not have access to the data because 

it had not previously had a supervisor for accountability and assessment to provide them 

with data. However, the district planned to provide professional development to teachers 

to help them understand how to use data for instruction and program evaluation: 

We are, I think, in the infancy stage of using data. Right now we’re fortunate to 
have a supervisor of accountability and assessment for the first time, which is 
really giving us access to data. So that was our first big step and now we have 
plans for next school year to provide staff development on how to use data, that’s 
our next big step. 

There was additional evidence that the district was using performance data to 

guide program planning and focus resources to address the achievement gap between 

minority ethnic groups and White students: 

Well, the data has been, in my opinion, extremely significant in general education 
in how we’ve allocated resources for minority kids, LEP kids, FARMS kids, 
because they are all below the standard and what do we do? So when it came 
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budget time I allocated additional dollars into programs and services that would 
positively impact those areas a bit more heavily, and targeted those schools that 
we have issues with or that needed additional help. 

However, looking at special education performance system-wide was in its initial 
stages. One of our informants reported: 

I took the special education data to the Board. They couldn’t understand why at 
first and said: “You have children with low ability so they are going to give us 
low performance.” So I just pulled out a chart of data. I said but here are the 
ranges of IQ. scores ‘cause they can understand IQs and here’s a 140 IQ kid with 
horrible scores. Why? Because of inappropriate instruction in reading and math, 
she was never taught to read. So I started coming at that then. 

Several infonnants at both the state and district level expressed frustration with 

the MSPAP. At the state level, informants expressed concerns over the validity of the 

MSPAP as a determinant of school effectiveness: 

We’re wrestling with some difficult issues in terms of accountability because of 
the great rewards and benefits to those schools that are showing progress. But is 
there real progress? Because they’re looking at the aggregated groups and all 
kinds of issues. 

At the district level, informants complained that they did not receive MSPAP 
scores in time to do anything about it instructionally: “One of the serious issues with 
MSPAP of course, is kids took the tests in May, we didn’t get the results back until 
November and we have a different group of kids.” Another complaint against the MSPAP 
was the lack of student level infonnation: “We firmly believe you cannot make an impact 
until you put a name to a number.” 
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Principals as instructional leaders. When we conducted our interviews at the 

state level in October 2001, informants stressed the need for school principals to become 

instructional leaders in their schools and provide cohesiveness for the whole school. The 

following quotation is one of many emphasizing the pivotal role of the principal, who had 

to be responsible for curriculum alignment with standards, hiring teachers based on the 

school needs, and leveraging the resources needed at the school level: 

The principal is the instructional leader. So if the principal doesn’t know the kind 
of performance expected of his or her staff then it’s a giant gamble that the staff is 
going to know as well. If you’re in a middle school and you’ve got 20 math 
teachers and they are all teaching 20 different targets the school system may have 
a curriculum in place, but the implementation is really at the principal level, the 
school level. Now it has to be the principal identifying what a teacher can and 
cannot do, and the principal working with that teacher saying this is the training 
you need, and the principal working with human resources saying this is the kind 
of teacher I need coming into my school, and these are the resources I need to 
train that teacher, and this is how I am supporting the growth of that teacher. You 
need a great principal who is the instructional leader, not the building manager. 

An MSDE informant reported that the state had implemented a program to 

support principals, but that they could only work with a fraction of the principals: 

We’ve now instituted a program to support principals in their professional 
development, but it’s a trial. I mean, you’re working with 200 or 300 principals, at 
most and we’ve got 1200 schools out there and 1200 principals, you’ve got tons 
that are on the margin. 

Another infonnant described another initiative aimed at principals. As the 
following quotation indicates, it required a paradigm shift from principals and schools in 
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that principals should look at what students were doing as opposed to what teachers were 
doing: 

We’re actually going to schools and working with principals. We’ll do a walk- 
through and we do not look at what is being taught; we’re going to look at what 
the kids are doing. So the focus is on student work and that is almost a paradigm 
shift for folks. Yet what we measure is what students do, not what teachers do. 

In this section, we discussed system level capacity to support the state’s 
educational reform agenda. It is clear that policy makers at the state and district levels 
especially were concerned over their lack of monetary and human resources. Policy 
makers at the state level were concerned about teacher shortages, the capacity of new and 
incumbent teachers to teach reading and mathematics, and use data, and the capacity of 
campus principals to become instructional leaders. Policy makers, at both the district and 
state level, recognized the inherent tensions between quality and quantity and had 
established several initiatives designed to address these challenges. In the following 
section, we look at the extent to which special education was included in the state’s 
reform agenda. 

Education Reform and Special Education 

School reform in Maryland was based on the premise that schools should be 
responsible for the learning of all students. Indeed, the state was ahead of most states in 
that it developed an alternate assessment for students with significant disabilities and was 
proactive in regard to the HSA and accommodations. However, when we conducted 
interviews with state informants in October 2001, there was considerable evidence that 
this population was not really a major focus of accountability reform and that special 
education as a system was struggling to be included in the ongoing reforms. Informants at 
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the state level identified several obstacles to overcome before special education could 
fully participate in accountability refonn. These obstacles included (a) ensuring all 
students with disabilities were included in the accountability system, (b) aligning special 
education accountability with regular education accountability, (c) issues around least 
restrictive environment, and (d) improving collaboration within MSDE. 

Including students with disabilities in the accountability system 

When Maryland developed its accountability system, it adopted an inclusive 
approach particularly in regard to participation in state assessments. For example, the 
IMAP assessment was developed for students with severe disabilities and students were 
allowed whatever accommodations were specified on their IEPs, because other than 
students on the IMAP, all students with disabilities were expected to participate in the 
MSPAP. Before the IMAP was developed IEP teams were allowed to exempt students 
with disabilities if the MLOs were not those students' identified instructional outcomes. 
Exempted students with disabilities were not included in the denominator for calculating 
the school's MSPAP scores. In addition, if prior to the MSPAP administration the school 
principal and at least one other qualified school staff member decided that testing would 
be severely harmful to a student, the student could be excused. A student could also be 
excused during the MSPAP if the testing became too stressful or the student became 
incapable of participating further. Excused students were included in the denominator for 
calculating the school's MSPAP scores. Finally, if a student received an accommodation 
that rendered a content area of the test invalid, the student became exempt and his or her 
score was not included in the denominator for calculating the school's MSPAP score in 
that content area. 
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When we conducted interviews with state level informants in September 2001, we 

found considerable evidence suggesting that districts did not really understand the 

accountability system, or the difference between exempt students and excused students: 

Right now we’re still having trouble. It’s just that each county is not doing it 
exactly, and when I go over the data with them, they start yelling: “This is 
wrong.” But if it’s wrong it’s because it’s coming from them. They’re just 
counting them wrong, so there’s still a confusion between excused and exempted. 
You’ll find more of the confusion at grade 3 than you will at grade - well, 5 is still 
iffy but at 5 it starts to straighten out and by 8 it’s pretty secure. But at grade 3 
you’ll find a lot of kids that we’re not sure how they’re counted. 

In addition, it appears that districts were unaware or did not consider the impact of 

accommodations on the validity of a student’s score and were providing accommodations 

whether or not the student needed them: 

There’s some counties that kind of have the unofficial policy of, you know, if 
you’re getting special ed in grade 3, the test should be read to you. So you see a lot 
of kids who are excused until we start looking at the numbers. Now they’ll say, 
“Well those kids participate in the test” but you see their scores aren’t reported. 

It’s a very different issue. 

The variability in how districts approached accommodations or determined which 
students with disabilities should be exempted created a significant problem for the state 
because the reliability of the MSPAP as a measure of school effectiveness was in doubt. 
As one informant commented: “Even though there’s data there, the likelihood of it being 
extremely accurate and reflective of what really goes on may not be.” MSDE infonnants 
reported that at a minimum they were trying to ensure that all students with disabilities 
were accounted for: 
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We need to lockdown to make sure no kid is being left, uninvolved in some test, 
statewide test. But there are real concerns because of the great rewards and 
benefits to those schools that are showing progress, etc. and then they’re saying: 
“But is there real progress?” because they’re looking at the aggregated groups and 
all kinds of issues. 

However, despite the problems discussed above, a dialogue between MSDE and 

the local school systems was beginning. Informants at the state level compared 

performance data between regular education students and special education students, 

pointing out that neither group gave a stellar perfonnance: 

Well, as you see, the general education students are not surging ahead either and 
their performance is terribly erratic. So if that were stable and moving upward, 
then we’d have a lot more accountability problems with why special education 
students are not also moving ahead. But obviously we are all working on pulling 
all students up and finding some consistent way of providing curriculum and 
instruction that make for change. 

Aligning special education accountability with regular education accountability 

Informants at the state level who were involved with accountability for special 

education reported that the state was trying to create more alignment with the changes 

that were taking place in regular education. The following quotation illustrates that 

special education monitoring was moving from process to outcomes and looking at data: 

It really and truly is a work in progress, but we’re trying not only to build a new 
system but a new way of operating which is kind of like MSPAP. I think the 
monitoring really is going to start driving more how the Division of Special 
Education does business, because it is data driven. 

The same informant went on to explain that the two systems needed to look at 
their data at the same time in order to plan for the following year: 
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Schools systems get their MSPAP results at the end of November. They usually, 
after the whooping and hollering or the tears are over, they start planning around 
January or February, some do it as late as March and April, for the following 
school year and that’s when special ed should do it, too. Special ed has always 
been one step behind, it seems like, or at least not in sync with the school system. 
When the school system looks at their exit data, looks at their graduation rate, 
does diploma versus certificate, that’s when special ed should be looking at their 
data. 

Under the new system, local districts were expected to look at their performance 

data, which were to measure the same variables that were important in regular education. 

The local school system would develop their improvement plans based on these data: 

Under perfonnance review we’re going to be looking at elements that have 
standards and elements that don’t. It’ll be much like the questions that are posted 
for general education. Who outperformed us? You know, so that they can literally 
do a search and see who in grade 3, with similar demographics, outperformed 
them. The idea is, where can I go to get some ideas? You know, to get some 
advice. 

However, as the following comment indicates, this new way of monitoring was 
very new to the counties and some were not prepared: “The counties that are just at the 
surface are just still wringing their hands. They haven’t gotten to those deeper level 
questions.” 

Issues around least restrictive environment (LRE) 

Several infonnants at the state level discussed issues relating to the least restrictive 
environment (LRE) in Maryland. Concerns arose in two specific areas: (a) the need to 
reduce the number of students with disabilities in separate settings, and (b) changing the 
conversation to access to the curriculum. 



56 




As the following statement indicates, LRE was a long term problem for MSDE as 
indicated by the Office of Special Education Programs (OSEP). According to this 
informant, a sizeable number of parents did not want their child to be placed in a regular 
education setting, despite the push for greater inclusion from OSEP and IDEA 97: 

“We’re looking at LRE. There’s a significant number of parents who couldn’t give a hoot 
and they want their kids in a special school. But the state, due to the drive from OSEP, is 
looking at LRE.” 

The same informant described a recent financial incentive from the state to 

encourage districts to place students in inclusive settings. As the following comment 

indicates, in the previous year the state gave districts $50 for every child placed in a 

regular education classroom: “Last year the State of Maryland gave everybody a financial 

incentive for kids to be in LRE A, which is the regular education classroom. For kids who 

were in those types of classrooms there was a $50 payment.” Another informant 

reiterated the state’s problems with increasing placement in more inclusive settings, but 

so far nothing had worked and LRE data were unchanged. As the following statement 

indicates, neither the state nor the federal government knew what to try next: 

LRE is stubborn and resistant to change in Maryland. Our data have been flat. We 
support millions of dollars into projects, initiatives, enhanced funding, incentives 
and just haven’t made much of an inroad in it and the Feds didn’t have much 
insight as to what we should do, and we just tried everything, everything. So other 
than shutting down self-contained, special schools. Short of that, I don’t know 
what else we’re going to do. 

In discussing LRE, several informants emphasized that the debate over LRE had 
changed as a result of standards-based refonn and IDEA 97 in particular. As the 
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following comment indicates, where a child was educated was becoming less important 

than whether the child had access to the general education curriculum and was 

demonstrating academic learning: “I’m not so souped-up on enviromnent; I’m souped-up 

on, you know — are kids learning to read and write? Access to the curriculum is very 

different than, you know, where they’re being taught.” As the following comment 

suggests, sometimes providing the necessary supports would require students with 

disabilities to be removed from the general education setting: 

Some districts are beginning to say, when we get kids early, in the early grades, 
we need to really immerse them, maybe give them double reading instruction and 
you don’t get that in general ed, so they’re going to be pulled out. But they’re also 
saying that they’re hoping down the road it may pay off because then you have a 
kid who can read, who can write, who can do some computation. Now they can be 
in general education. 

Informants also raised an interesting point for accountability and resource 

allocation, about the need to determine whether the resources expended in getting 

students into the regular setting were effective in improving performance. As the 

following response indicates, there is little hard data to suggest that Maryland students in 

inclusive classrooms perform better than similar students placed in separate classrooms: 

We have been lobbying real hard to get something on MSPAP so that we can see 
the financial money that we’re putting toward kids being in a general education 
class actually had some type of effect. Everybody assumes that the kids are going 
to do better in a general education class, but there is no data that we have that says 
that special education students in a general education class that take MSPAP do 
better than students with disabilities that are in separate classes that take MSPAP. 
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Improving collaboration within MSDE 

If accountability reform is to be successful for all students, then ideally the needs 

of all students will be considered when the components of accountability are put in place. 

This requires that those responsible for each component — instruction and curriculum, 

accountability, and assessment — collaborate with each other and other divisions as 

necessary, such as special education and bilingual services. However we found 

considerable evidence suggesting that the level of collaboration was problematic: 

Well now we say to them “Oh, what are you doing?” And they say, “Oh no, we 
haven’t included special education in that you know.” It should be, “How do we, 
as a division, get special education included up front?” 

Several infonnants commented that the level of collaboration between special 
education and other components of MSDE was not as good as it needed to be. Indeed, 
one informant characterized the lack of collaboration as a major barrier to the inclusion of 
students with disabilities in accountability reform: “One of the things we didn’t talk about 
is coordination between individuals here, and there isn’t any, which I see as a big issue. 
It’s a big problem.” 

As the following statement indicates, the level of alignment was particularly low 

between the Division of Instruction and the Division of Special Education for the 

individuals who had to implement the policies: 

We need greater alignment with the Division of Instruction, because we don’t 
have that. So we’re running around trying to figure out what they just did. Just all 
of a sudden they have an impact on us and we find out from the locals. There is 
no coordination here that I can see. I know the Associates sit around, but it 
doesn’t come down to the worker bee level, and I consider myself a worker bee. 
That’s not the way to do business. It doesn’t give us any credibility. So that to me 
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is probably the biggest barrier. 



However, the same informant indicated that the situation was slowly beginning to 
change. This individual attributed this change to the fact that special education was 
beginning to speak the same language as regular education, and not simply concerning 
monitoring for compliance: 

They’ve only seen us as doing compliance monitoring. They’ve never seen us do, 
you know, program perfonnance and looking at outcomes. Now that we’re 
beginning to speak their language, or they know that we have an interest in what 
they’ve an interest in, they’re much more receptive. 

In this section, we have looked at the extent to which students with disabilities are 
meaningfully included in accountability reform. Although the state considered the needs 
of students with disabilities in terms of their actual participation in the assessment, 
informants expressed concerns over the leeway provided to IEP teams to exempt or 
excuse students with disabilities from state assessments. Informants were also concerned 
about the validity of decisions made about school effectiveness when districts interpreted 
state policies differently and schools were judged on aggregated data. Informants 
provided evidence that special education as a system was in greater alignment with 
accountability in regular education and was beginning to look at the same indicators and 
work toward the same outcomes. However, informants were also grappling with the issue 
of LRE, though the conversation was beginning to move from inclusion to access to the 
curriculum, perhaps in contrast to OSEP’s concerns. 
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V. MARYLAND CASE STUDY DISCUSSION AND CONCLUSION 



In the previous section, we presented qualitative data, collected over a two-year 
period from two levels of the public education system, on the impact of accountability 
reform on students with disabilities in Maryland. State level interviews took place in 
September 2001, while district interviews took place in June 2002. The sequential nature 
of data collection enabled us to study the impact of state accountability reform on 
students with disabilities and to follow the concerns identified by state level policy 
makers through to the districts. However, collecting data in this way meant that we were 
unable to obtain the initial views of state level personnel on the impact of NCLB, signed 
into law after the completion of the state level interviews, or to study the further 
development and refinement of federal accountability policy for this population. When 
we visited the districts in the following year, there was similarly little discussion of the 
impact of NCLB legislation. 

EPRRI staff interviewed 10 individuals, 7 at the state level and 3 at the district 
level. Findings are arranged into three broad topical areas: (a) upcoming changes to the 
accountability system, (b) building instructional capacity, and (c) reform in special 
education. When considering the impact of standards-based accountability reform on 
students with disabilities in Maryland, it is important to remember that the state had an 
established accountability system for some time. The current wave of reform at the state 
level continued an already established path that aimed to improve student performance by 
changing what students learned and how teachers taught. However, changes in federal 
education reform, although supporting the inclusion of students with disabilities, were 
problematic for the Maryland system in certain key ways. In addition, accountability for 
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student learning was a recent concept for schools and districts. With this mind, the 
following key findings emerged from this study. 

First, several informants reported that there was a lack of alignment at the local 
level between curriculum and instruction and the state learning standards. For instance, 
some reported that while the state had developed content standards in grade bands, the 
scope and sequence across content areas was left up to individual schools systems, 
resulting in a good deal of variability across schools. In addition, informants stated that 
there was some confusion at the school level regarding the MSPAP. Specifically, teachers 
were unclear about what the test actually measured and did not know how to prepare their 
students for the test, leading to inconsistency between instruction and what the students 
were expected to do on the MSPAP. 

Other findings that surfaced were related to the introduction of the HSA. In 
particular, informants commented that they were concerned about the emergence of a 
group of students with disabilities who were capable of meeting all of the high school 
graduation requirements, but would not be able to pass the HSA. In addition, informants 
expressed concern over the fact that these students would not be able to enter many 
postsecondary schools without a high school diploma, even though they had the potential 
to succeed in these settings. However, informants also reported that they were 
considering alternative options for these students, including allowing them to earn a 
program completion certificate or an IEP diploma. Other concerns were raised about the 
placement of students with disabilities and how these placements affected the students’ 
ability to access the curriculum and instruction they would need to pass the HSA. A final 
theme that emerged from the state and district level interviews regarding the introduction 
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of the HSA was the need for certified teachers to provide the instruction necessary to help 
students succeed on the HSA. 

Findings also emerged concerning the state system after the reauthorization of 
ESEA. Specifically, several district level informants expressed frustration at the lack of 
information they were given concerning the new assessment system intended to replace 
MSPAP and any changes this new test would require at the curriculum level. This 
uncertainty required school systems to be reactive rather than proactive, as they would 
not be able to implement any changes before the school year commenced. 

Financial and human resource concerns also emerged as significant findings. 
Informants at the state and district levels reported that they were experiencing significant 
budget shortfalls that impeded both their ability to support the state’s reform agenda and 
their ability to respond to changes in the assessment system. Informants also 
communicated that Maryland was experiencing a shortage of experienced teachers and 
administrators. State level informants also commented on the inability of districts to 
move successful educators and administrators into struggling schools, stating that the 
lowest performing schools were those that had the least experienced teachers and the 
highest turnover of teachers. Some informants discussed their attempts to address this 
issue and pointed out that districts were unable to control where teachers taught due to 
seniority and mobility options within the district. Informants also described various 
district incentives that were designed to encourage teachers to teach in struggling schools, 
such as increasing teacher salaries and creating 1 1 -month contracts. Despite these efforts, 
informants reported that the shortage of teachers and administrators remained. 

Other findings related to the need to improve educator knowledge and skills. In 
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particular, informants reported the need to improve the quality of new teachers entering 
the profession by increasing teacher education standards at colleges of education and 
increasing the rigor of the licensing requirements. However, they also stated that there 
was little funding from the state to finance these improvements. Informants also 
described the Maryland initiative and its goal to close the achievement gap between 
minority and non-minority students by exposing teacher candidates to a diverse student 
population. Other comments related specifically to special education teacher preparation 
programs and in particular, the need for special education teachers to have more content 
area knowledge. Informants discussed the need to improve the skills of incumbent 
teachers as well, and described the state requirements for every county and teacher to 
have professional development plans in place and approved as a requirement for teacher 
certificate renewal. However, informants reported that many school systems did not have 
staff development plans for special educators. 

Several infonnants at the state level reported that improving educator knowledge 
and skills at the middle school level was an area of particular concern. More specifically, 
informants stated that many middle school mathematics teachers did not have the content 
area skills they needed to instruct their students. Other infonnants reported that middle 
school teachers needed to be better prepared to help struggling readers. Finally, in regard 
to the area of improving educator knowledge and skills, informants discussed the need to 
develop the capacity of districts and teachers to use data in making instructional 
decisions. In particular, they reported the need for the state to use data to address the gap 
between regular education students and special education students. 

Additional comments related to state and district capacity to support reform 
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concerned the need for school principals to become instructional leaders in their schools 
and provide cohesiveness for the whole school. Specifically, many informants described 
the pivotal role of the principal, who needed to be responsible for curriculum alignment 
with standards, hiring teachers based on school needs, and leveraging the resources 
needed at the school level. Informants at the state level did recognize the need for 
programs to support principals, although they commented that they only had the 
resources to work with a fraction of the principals. 

The third theme that emerged from interviews at the state and district levels dealt 
with education reform and special education. Comments from informants at both levels 
provided considerable evidence that the special education population was not really a 
major focus of accountability reform and that special education as a system was 
struggling to be included in ongoing reforms. In addition, comments from state level 
informants indicated that district personnel did not fully understand the difference 
between exempt students and excused students nor the impact of accommodations on the 
validity of a student’s MSPAP score and moreover were providing accommodations 
whether or not the student needed them. As a result, the reliability of the assessment as a 
measure of school effectiveness was in doubt. 

Other comments regarding education reform and special education indicated that 
there was a need to better align special education accountability with regular education 
accountability. However, infonnants also stated that they were still grappling with 
longstanding concerns related to LRE in Maryland as OSEP required the state to reduce 
the number of students with disabilities placed in separate settings. Interestingly, 
informants reported that the debate over LRE had begun to change, and where a child 
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was educated was becoming less important than whether the child had access to the 
general education curriculum and was demonstrating academic learning. Related to this 
was the need to obtain performance data to compare academic growth in inclusive 
classrooms with that in separate settings for similar students. Finally, informants 
discussed the need for improving collaboration within MSDE divisions such as 
curriculum and instruction, accountability, assessment, and special education and 
bilingual services in order to increase the extent to which students with disabilities are 
included in accountability reform. 
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